Neural Combinatorial Deep Reinforcement Learning for Age-Optimal Joint Trajectory and Scheduling Design in UAV-Assisted Networks

In this article, an unmanned aerial vehicle (UAV)-assisted wireless network is considered in which a battery-constrained UAV assumed to move towards energy-constrained ground nodes receive status updates about their observed processes. The UAV’s flight trajectory and scheduling of are jointly optimized with the objective minimizing normalized weighted sum Age Information (NWAoI) values for different physical processes at UAV. problem first formulated as mixed-integer program. Then, given policy, convex optimization-based solution proposed derive optimal time instants on updates. However, finding policy challenging due combinatorial nature problem. Therefore, complement solution, finite-horizon Markov decision process (MDP) used find policy. Since state space MDP extremely large, novel neural combinatorial-based deep reinforcement learning (NCRL) algorithm using Q-network (DQN) obtain large-scale scenarios numerous nodes, DQN architecture cannot efficiently learn anymore. Motivated by this, long short-term memory (LSTM)-based autoencoder map fixed-size vector representation such while capturing spatio-temporal interdependence between update locations instants. A lower bound minimum NWAoI analytically derived provides system design guidelines appropriate choice importance weights nodes. Furthermore, upper speed obtained achieve value. numerical results also demonstrate that NCRL approach can significantly improve achievable per compared baseline policies, weight-based discretized policies.